DOCUMENT RESUME 



ED 426 287 

AUTHOR 

TITLE 



PUB DATE 
NOTE 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 
ABSTRACT 

education (VE) teachers use student assessment information in making 
instructional decisions. A 5-part questionnaire designed to determine how 6 
types of student assessment methods were being used in addressing 10 
instructional decisions was administered to a randomly selected cluster 
sample of 240 of all 647 West Virginia teachers who taught full time in 
1997-1998. Of those surveyed, 144 (60%) returned usable questionnaires. 
Multiple regression analysis was used to determine the variance in the use of 
assessment information as explained by selected independent variables. The 
respondents, who averaged 15.02 years of teaching experience and 9.66 years 
of related work experience, considered information generated from performance 
assessment the most useful type of assessment information. The remaining five 
sources of assessment information was ranked as follows (from most to least 
important): informal observation, objective test items, standardized test 
scores, essay items, and portfolios. Results suggest that secondary VE 
teachers need the following in order to make changes that are conceptually 
meaningful: appropriate materials to try and adapt; time to reflect and 
develop new instructional approaches, and ongoing support from experts to 
learn the conceptual bases behind intended reforms. (Contains 18 references.) 
(MN) 



CE 077 994 

Gordon, Howard R. D. 

Influence of Selected Variables on the Use of Different 
Assessment Methods as Perceived by Secondary Vocational 
Education Teachers. 

1999-02-00 

32p. ; Paper presented at the Annual Eastern Educational 
Research Association Conference (22nd, Hilton Head, SC, 
February 24-27, 1999) . 

Reports - Research ( 143 ) - - Speeches/Meeting Papers ( 150) 
MF01/PC02 Plus Postage . 

♦Information Utilization; Secondary Education; * Secondary 
School Teachers; State Surveys; Statistical Analysis; 
♦Student Evaluation; ♦Teacher Attitudes; ♦Test Results; 
♦Vocational Education 
♦West Virginia 



A study examined how West Virginia secondary vocational 



******************************************************************************** 



♦ Reproductions supplied by EDRS are the best that can be made ♦ 

♦ from the original document. ♦ 

******************************************************************************** 




Selected Variables 



r~- 

00 

<N 

VO 

<N 



Q 



W 



Running head: INFLUENCE OF SELECTED VARIABLES 



Influence of Selected Variables on the Use of Different Assessment Methods as 
Perceived by Secondary Vocational Education Teachers 



Howard R. D. Gordon 



Marshall University 



L.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
7 CENTER (ERIC) 

(□/This document has been reproduced as 
V received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



Huntington, West Virginia 25755 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 



• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



e- 

(N 

O 



V) o 




Paper presented at the 22"^ Annual Eastern Educational Research Association Conference 

Hilton Head, South Carolina 



February 24 - 27, 1999 



2 



BEST COPY AVAILABLE 



Selected Variables 



2 



Abstract 

The purpose of this study was to describe West Virginia secondary vocational education 
teachers’use of student assessment information in making instructional decisions. The 
study examined the use of assessment information, obtained from six types of student 
assessment methods, in addressing 10 instructional decisions. The target population was 
all teachers in West Virginia who taught frill time at secondary technical centers during 
the 1997-98 school year. A cluster sample of 240 teachers was used for the study. A five- 
part questionnaire was designed to measure selected variables. Descriptive statistics were 
used to summarize the data. Multiple regression analyses were used to determine the 
variance in the use of assessment information as explained by selected independent 
variables. Respondents had an average of 1 5 years of teaching experience. Secondary 
vocational education teachers rated information provided from performance assessments 
as being of more use in addressing daily classroom decisions than information obtained 
from the other five methods. To make changes that are conceptually meaningful, 
secondary vocational education teachers need: appropriate materials to try out and adapt, 
time to reflect and to develop new instructional approaches, and ongoing support from 
experts to learn the conceptual bases behind intended reforms. 
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Influence of Selected Variables on the Use of Different Assessment Methods as 
Perceived by Secondary Vocational Education Teachers 

During the past few years, economic concerns have prompted a number of 
proposals aimed at reforming federal vocational education and employment training 
programs. Reports about the inadequate skills of high school graduates, the rapidly 
changing demands of the workplace, and the declining competitiveness of U.S. firms in 
the international marketplace have all fueled the idea that the organization and structure 
of employment preparation programs must change. The 104* congress argued at length 
about this issue, but Democrats and Republicans were not able to reach a consensus about 
the shape of future federal vocational education programs. 

However, despite the strong differences in their approaches to reform, all sides 
seem to agree on the need for trustworthy methods for assessing vocational students’ 
skills. A noteworthy example of this convergence can be seen in the continuing debate 
between those who recommend a greater focus on broad industry skills at the secondary 
level (Boesel & McFarland, 1994) and those who place more emphasis on occupation- 
specific skills (Bishop, 1995). Both sides agree about the need for a system that assesses 
skills in reliable and valid ways. Almost all policy makers think it is essential to measure 
the degree to which participants have mastered the skills upon which training focuses. 

Many vocational educators are advocating the wider use of alternative 
assessments, such as portfolios, exhibitions, and performance events, for measuring skills 
of either type. This interest in new measures derives in part from the changes occurring 
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in vocational education. Educators and employers believe that the work world is 
changing and vocational education must adapt if it is to serve students well. The changes 
in the workplace are complex and not completely understood, but most observers believe 
that future employees will need integrated academic and vocational knowledge, a broad 
understanding of occupational areas, the ability to interact creatively with their peers, and 
higher-order cognitive skills that allow them to be flexible, learn rapidly, and adapt to 
ever-changing circumstances. To the extent this belief is true, vocational training needs 
to place greater emphasis on integrated learning, critical thinking skills, and connections 
between vocational and academic skills, rather than on the mastery of narrow, 
occupation-specific skills that characterized vocational education in the past. This new 
vision may also require broader changes in vocational education, including rethinking the 
organization, goals, content, and delivery of services, as well as the manner in which 
students and programs are assessed (Stecher, et al., 1997). 

Theoretical Framework 

The American Federation of Teachers, the National Council on Measurement in 
Education, and the National Education Association (1990) supported the concept of an 
instructional-assessment linkage by subscribing to the view the “student assessment is an 
essential part of teaching and that good teaching cannot exist without good student 
assessment. ”(p 1) 

Door-Bremme (1983) stated that if testing programs were to be useful for teachers 
they must take into account teacher’s routine thinking and practices in assessing student 
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achievement. This study also found that the type of assessment methods that teachers 
rely upon most heavily have three common characteristics; 

1 . Immediate accessibility; teachers can give them when they choose and see 
results promptly. 

2. Proximity between their intended purposes and teachers’ practical activities. 

3. Consonance between the content they cover and the content taught, (p. 5) 
Door-Bremmer (1983) concluded that teachers perceive their use of assessment methods 
as accurately measuring the effects of the instruction they provided. 

Shulman (1980) believed that a teacher’s ability to conduct an array of assessment 
practices should generate a rich source of useful data for decision making. Shulman 
(1980) made reference to the assessment practices of a physician in emphasizing the 
range of activities that a teacher could be expected to perform. 

The physician uses observation, interview, touching and feeling, as well as 
testing, and develops an assessment and a plan by aggregating across those 
sources of information rather than by giving almost total weight to any one 
source and subordinating the others to it. (p. 69) 

The demographic and background characteristics of teachers have been 
recognized as factors which can explain variations in teacher competence and particular 
teacher use of assessment (Kershaw, 1993). Several teacher background characteristics 
were found by Newman and Stallings (1982) to be correlated with classroom assessment 
competency. It was determined that younger teachers, teachers with higher degrees, and 
teachers with less teaching experience tended to score higher on an instrument which 
measured an understanding of classroom assessment principles. 
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State after state has begun to initiate mechanisms, which serve to promote 
accountability for educational outcomes. California, Connecticut, Iowa, Kansas, 
Maryland, New Jersey, and Ohio have taken measures to make their educational systems 
more accountable for student outcomes (McCaslin, 1990; Kershaw, 1993), Pressure has 
been mounting for vocational educators to become more accountable for learning 
outcomes of their students. How teachers use assessment information in the classroom 
and whether its use is effective can play a major role in enhancing and documenting both 
instruction and learning. How are vocational education teachers using student assessment 
data? The literature has revealed very little information about the assessment practices of 
this group of teachers. How vocational education teachers use assessment data must be 
described if we are to improve critical components of the assessment process. 

Purpose and Objectives 

The purpose of this study was to describe West Virginia secondary vocational 
education teachers’ use of student assessment information in making instructional 
decisions. The specific objectives of the study were as follows: 

1 . To describe secondary vocational education teachers’ perceptions of their 
use of student assessment data for making instructional decisions. 

2. To describe secondary vocational education teachers’ perceptions of their 
competence, constraints and attitudes toward the assessment process. 

3. To determine the proportion of variance in secondary vocational education 
teachers’ perceived use of assessment information in instructional decision 
making that could be explained by the independent variables of attitude 
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towards assessment, competence in the assessment processes, constraints 
to the assessment process, and selected demographic and background 
characteristics ( gender, age, years of teaching experience, years of related 
work experience, vocational teaching area, educational level, and 
certification route). 



Method 

Population and Sampling 

The target population (N = 647) was all teachers in West Virginia who taught fiill 
time at secondary vocational technical centers during the 1997- 98 school year. The 1997 
- 98 West Virginia Education Directory served as the sampling fi'ame for the study. 

Using Krejcie and Morgan’s (1970), table of sample sizes, a sample size of 240 was 
identified as representative of a population of 647 within a five percent margin of error. 

A cluster sampling technique was used to randomly select secondary vocational 
education teachers from the 32 secondary vocational technical centers in the state of West 
Virginia. Twelve secondary vocational technical centers were randomly selected to 
participate in the study in order to achieve the desired sample size of 240 (12 schools 
with an average of 20 teachers per school). Secondary vocational technical centers were 
numbered from 1 to 32, and the 12 schools were selected using the random number 
generation in Microsoft Excel. 

Cluster sampling is appropriate to use in educational settings when it is 
impossible to select a random sample of individuals (Fraenkel & Wallen, 1996). 
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However, the disadvantage of cluster sampling is that each stage of the process increases 
sampling error. 

Instrumentation 

The five-part survey used in this study was adapted from a survey used in a 
similar investigation conducted by Kershaw (1993). A 5-point Likert-type scale was 
used throughout the instrument except for the measurement of attitudes toward 
assessment. Muller (1986) stated that using a scale with a middle category seems to work 
as well as a scale without a middle category. 

The use of assessment information was measured in part one. Teachers were 
asked to indicate the extent they used information from six types of assessment methods 
in addressing ten different instructional decisions. The six types of assessment methods 
used in this study were (1) objective paper and pencil items, (2) informal observations, 

(3) standardized test scores, (4) performance assessments, (5) portfolios, and (6) essay- 
type items. The 10 instructional decisions that were addressed were (1) planning 
instruction, (2) diagnosing student weaknesses, (3) monitoring student progress towards 
course objectives, (4) communicating student achievement with parents, (5) motivating 
students to learn, (6) evaluating the effectiveness of instruction, (7) evaluating 
instructional materials used, (8) grouping students for instructional activities, (9) 
encouraging students to assess their own work, and (10) assigning grades. The following 
Likert scale was used: 1 = of no use; 2 = of limited use; 3 = of some use; 4 = of much 
use, 5 = of considerable use. 
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Part two was designed to measure participants’ perceived level of competence in 
the assessment process. Competency statements were based upon “Standards for 
Teachers, National Council on Measurement in Education, and National Education 
Association, 1990). The following Likert scale was used; 1 = not competent; 2 = slightly 
competent, 3 = moderately competent; 4 = very competent; 5 = extremely competent. 

Participants’ attitudes toward assessment was measured in the third part of the 
instrument using a semantic differential scale. The scale consisted of nine bipolar 
adjectives (worthless - valuable, unsuccessful - successftjl, inefficient - efficient, 
unimportant - important, bad - good, unfair - fair, disreputable - reputable, rigid - 
flexible, and tense - relaxed) that represented various assessment -related concepts. A 7- 
point scale was used for each pair of adjectives. The scale was as follows: 1 = extremely 
negative; 2 = very negative; 3 = negative; 4 = neutral; 5 = positive; 6 = very positive; 7 = 
extremely positive. Respondents were asked to place a check mark between the 
adjectives at the point that best indicated how they would describe their attitude towards 
the overall assessment process. 

In part four of the questionnaire, nine statements were used to measure teacher’s 
perceptions of constraints to their assessment activities. Major constraints identified in 
the literature included: time, money, technology and assistance, training, autonomy in 
making assessment-related decisions, and availability of assessment materials. The 5- 
point Likert scale used in this section ranged from strongly disagree to strongly agree. 
Participants were asked to indicate their level of agreement with each of the constraint 
statements. Part five of the instrument consisted of demographic questions. 
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To ensure validity of the instrument, a panel of experts was used to establish 
content and face validity. The panel consisted of three vocational education teachers, a 
regional teacher educator, and two professors of vocational education. The instrument 
was field-tested for reliability with a sample of vocational education teachers (n = 1 1) not 
selected for participation in the study. Changes indicated by the validation panel and field 
test were made. Internal consistencies for the scales in the instrument were as follows 
(Cronbach’s Alpha): Use of Assessment .94, Competency in Assessment .93, Attitudes 
Toward Assessment .95, and Constraints in Assessment .64, acceptable according to 
Nunnally and Bernstein (1994). 

Data Collection 

The Total Design Method (TDM) of conducting surveys (Dillman, 1978) was 
followed in all stages of the questionnaire construction and implementation process. A 
packet containing a cover letter, instructions for administering the questionnaire, and 
copies of the questionnaire were mailed to the principal of each school selected for the 
study. A total of 240 questionnaires was sent to 12 principals during October, 1997. A 
total of 144 usable questionnaires was returned for a response rate of 60%. Because a 
number of questionnaires were returned uncompleted and there was no way to conduct 
appropriate follow-up procedures to control for non-response, the results of the study can 
only be generalized to the 144 teachers who provided usable data. 
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Data Analysis 

Data were analyzed using the Statistical Package for the Social Sciences (SPSS 
Version 6. 1 for Windows). Descriptive statistics were used to summarize the data. 
Multiple regression analyses were used to determine the variance in the use of assessment 
information as explained by selected independent variables. An alpha level of .05 was set 
a priori for the study. 



Results 

Demographic tpharacteristics 

The Iwgest number of teachers in the sample fell within the 42-51 year age 
bracket (52. 1%) of the 144 cases, 56% of the respondents were male and 44% were 
female. Teachws had an average of 15.02 years of teaching experience and 9.66 years of 
related work experience. Trade and Industrial Education teachers comprised the largest 
group in the sample and accounted for 43% of the cases. A graduate degree had been 
earned by 39% of the respondents. Respondents who had completed a teacher preparation 
program on the job, and before receiving a bachelor degree, comprised 43% of the cases 
(see Table 1). 

Use of Assessment Information 

Table 2 depicts teachers’ use of assessment information and educational decision 
making. Information generated from performance assessment was considered to be of 
more use to teachers in addressing educational decisions than any of the other assessment 
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methods (M = 4 . 19). Teachers rated performance assessment information to be of much 
use (M = 4 .45) when specifically addressing the task of assigning grades. 

Teachers reported that information derived from informal observations (M =3 -96) 
was of more use than all other assessment methods except performance assessment. 
Informal observations provided information that was considered by teachers to be of 
much use (M “= 3 .80-4 .46) when addressing all 10 decision areas. 

Information obtained from objective paper and pencil items ( M = 3 .91) were 
found to be of less use than both performance assessments and informal observations, 
but of more use than essay items, portfolios, and standardized test scores. Information 
from this assessment method was found to be of much use (M = 3 .68 - 4 . 14) for nine of 
the decision areas. Teachers revealed that objective and paper pencil items were of some 
use when grouping students for instructional activities. 

Portfolio assessment, as defined by Vavnis, (1990) is a systematic and organized 
collection of evidence used by the teacher and student to monitor growth of the student’s 
knowledge, skills, and attitudes in a specific subject area. It is a collection of student 
work which reflects student achievements over time. A portfolio contains documentation 
of not only the products that are generated by the student but also the processes that are 
involved. It might contain drawings, written documents, tests, notes, photographs, or 
comments from teachers and peers. 

Portfolios were found to be of only some use (M 2 .81) in providing 
information in the overall decision making process. However, teachers found portfolios 
to be of some use (M ~ 3 .02) when specifically communicating student achievement 
with parents. 
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Standardized test scores (M = 2 .75) and essay-type information (M = 2 .72) were 
the two assessment methods found to be of less importance to teachers when compared to 
the other methods. Teachers reported that standardized test scores were of some 
use when evaluating the effectiveness of their teaching methods (M = 3 .03) and of 
limited use (M “ 2 .37) when assigning grades. For addressing educational decisions, 
information obtained through the implementation of essay type methods were found to be 
of some use for all ten decision areas (M= 2 .50 - 2 .84). 



Insert Table 2 about here 



Perceived Level of Competency in Assessment 

Table 3 provides information regarding teachers’ perceived level of competence 
in individual assessment activities. Teachers considered themselves to be very competent 
(M = 4 .35 - 3 .52) in all but three areas of the assessment process. Teachers perceived 
themselves to be only moderately competent when interpreting the scores of standardized 
achievement tests (^= 3 .24), scoring an essay (M = 3 .36), and compiling a student 
portfolio (M = 3 .38) 



Insert Table 3 about here 



Attitudes Toward Asessment 
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A semantic differential scale was used to measure teachers’ attitudes toward the 
overall assessment process. Teachers were asked to respond to nine pairs of bipolar 
adjectives which were on either end of a 7-point scale. Respondents reported that they 
perceived the assessment process to be valuable, efficient, and important ( M = 5 .77 to 5 
.40). The assessment process was perceived as less positive in terms of being fair, good, 
reputable, successful, flexible, and relaxed ^ = 5 .37 to 4 .46). 

Constraints to the Assessment Process 

Table 4 describes the responses to each of the constraint questions. Teachers 
tended to agree that they decided on what assessment methods to use in their courses (M 
= 4 .28). Teachers also tended to agree that additional planning time would allow for 
assessment methods to be used more effectively = 3 .94). Over two-thirds of the items 
were rated as “neutral” by respondents in this study (M = 2 .69 - 3 .00). 



Insert Table 4 about here 



Multiple Regression Models 

Table 5 reveals that 1 5% (R^ = . 1 54) of the variance in the use of objective paper 
and pencil assessment methods was due to the combined linear effects of the independent 
variables. Two variables: “competence in assessment” and “attitudes toward assessment” 
were shown to be significant contributors of objective paper and pencil assessment 
methods. 
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Insert Table 5 about here 



The regression of the use of standardized test scores on selected independent 
variables revealed that the proportion of variance explained was 0. 1 % .001). This 

model was statistically not significant (p > .05). 

The third statistical model regressed the dependent variable, use of performance 
assessment, on selected independent variables. The independent variables accounted for 
21% of the variance. Attitude, constraints, gender, and certification were significant 
contributors of the use in performance assessment, (see Table 6). 



Insert Table 6 about here 



Table 7 summarizes the regression of the use of the informal observations on 
selected independent variables. It was found that attitudes toward assessment, 
competence in assessment, and certification were significant contributors in the use of 
informal observations. The model was statistically significant (F = 3 .768) 



Insert Table 7 about here 



The regression of the dependent variable, use of portfolios, on selected 
independent variables revealed that the proportion of variance (R^ .086) explained was 
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statistically not significant (p > .05). Table 8 shows the regression model for essay-type 
items. Only competence and certification combine to be significant contributors of essay- 
type items. This model was statistically significant (E = 2 .33). 



Insert Table 8 about here 



Conclusions 

The results of this study cannot be generalized beyond the respondents to this 
survey. The typical secondary vocational education teacher 

• is more likely to be in the age bracket of 42-5 1 years old; 

• completed an average of 1 5 years of teaching and 10 years of 
related work experience, 

• completed a graduate degree, 

• completed a teacher preparation program on the job; and 

• is more likely to be a teacher in trade and industrial education. 
Some of these findings are partially explained in a study done by Lynch (1993). 
According to Lynch’s study (1993), secondary vocational education teachers tend to have 
less formal education than others, but they have more related occupational experience 
and credentials. This emphasis on occupational experience in lieu of formal education is 
concentrated in trade and industrial education, where it has been guided by state policies 
in a tradition going back to back to Smith-Hughes Act of 1917. 
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Teachers rated information provided from performance assessments as being of 
more use in addressing daily classroom decisions (example, when specifically addressing 
the task of assigning grades) than information obtained from the other five methods. This 
finding suggests that performance assessments are particularly useful to vocational 
educators because they can be used to stimulate real occupational settings. This finding 
is supported by a study done by Kershaw (1993). According to Kershaw’s study, 
secondary vocational education teachers in Ohio were more likely to use performance 
assessments than any other methods. 

Attitudes toward assessment were viewed as being positive by secondary 
vocational education teachers. Teachers with more positive attitudes toward assessment 
and with higher levels of competence in assessment tended to use performance 
assessments more than teachers with less positive attitudes and weaker levels of 
competence in assessment. Scharfer and Lissitz (1987) concluded that although teachers 
may be ill-trained to use accepted measurement practices, they see assessment as an 
important part of their professional role and feel positively toward it. 

There appeared to be little practical variance explained with essay items, 
portfolios, and standardized tests in the use of assessment methods. Demographic and 
background characteristics accounted for no significant variation in teachers’ use of 
assessment methods. These findings are in agreement with a study done by Kershaw 
(1993). However, this study contradicts findings by Ryans (1960); Yeh, Herman, and 
Rudner (1981). Ryans (1960) found significant differences in teacher’s classroom 
behavior due to variables such as age, and teaching experience. Yeh, et al. (1981) 
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reported that years of teaching experience was found to be related to different patterns of 
assessment. 



Implications 

To select an appropriate assessment a teacher must know how the information 
elicited from it will be used, since high stakes assessments used for placement decisions 
require much more rigor in their development to ensure high reliability and validity than 
do assessments used for low stakes purposes. Assessments designed to give students 
feedback for developmental purposes do not have to meet the same level of statistical or 
psychometric rigor that decision-making assessments do. Thus, any of the assessment 
types described in this study can be used effectively in the classroom for low stakes or 
medium stakes purposes. Teachers have considerable latitude in how they assess 
students’ knowledge and skills, but they need to think about how assessment can 
contribute to students’ learning about themselves, particularly in relationship to skills 
needed in the workplace. 

The moderate levels of perceived assessment competence as documented in this 
study, the small but significant influence with the use of assessment methods, along with 
research which has identified the deficiencies of teacher assessment skills, suggest the 
potential necessity or the upgrading of secondary vocational education teacher 
competence in assessment practices. 

To make changes that are conceptually meaningful, secondary vocational 
education teachers need: appropriate materials to try out and adapt; time to reflect and to 
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develop new instructional approaches, and ongoing support from experts to learn (and 
challenge) the conceptually bases behind intended reforms. 
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Table 1. 

Categorical Demographic Information (n = 144) 



Variable of Interest 


Frequency 


Percent 


Gender: 


Male 


81 


56 


Female 


63 


44 


Age : 


22-31 


4 


2.8 


32-41 


31 


21.5 


42-51 


75 


52.1 


52-62 


32 


22.2 


63 and above 


2 


1.4 


Years of Teaching Experience: 


0-10 


55 


38.20 


11-21 


49 


34.02 


22-23 


38 


26.39 


33 


2 


1.39 


Years of Related Work Experience: 


0-5 


50 


34.72 


6-11 


4 


31.94 


12-17 


24 


16.66 


18-23 


11 


7.63 


24-29 


9 


6.25 


30 


4 


2.80 


Vocational Education Program Areas: 


Agriculture 


6 


4.2 


Business Occupations 


13 


9.0 


Health Occupations 


14 


9.7 


Family and Consumer Sciences 


9 


6.2 


Marketing 


1 


0.7 


Trade and Industrial 


62 


43.1 


Other Related Areas 


39 


27.1 



(table continues ) 
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Table 1 continued 



Variable of Interest 


Frequency 


Percent 


Highest Educational Level: 






High School Diploma 


1 


0.7 


Some College 


30 


20.8 


Associate Degree 


10 


6.9 


Bachelor’s Degree 


47 


32.6 


Master’s Degree 


56 


39.0 


Certification Route: 






Teacher preparation program 
completed prior to teaching 


59 


41 


Teacher preparation program 
completed after entering teaching 
and before receiving a degree 


62 


43 


Teacher preparation program 
completed after entering and 
teaching and after receiving 
a degree 


23 


16 
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Tables. 



Process (0=144)* ** 



Characteristics M SD 



Grade objective paper and penci 1 items 4,35 .76 

Score a performance assessment 4.14 .82 

Prepare students to take tests 4. 12 .74 

Administer a performance assessment 4. 1 1 .85 

Communicate assessment result to students 4.07 .85 

Develop student grading procedures 4.06 .86 

Select methods for assessing student performance 4.05 .79 

Match items to intended learning outcomes 4.03 .83 

Select assessment methods for monitoring student learning 3.93 .76 

Communicate assessment result to employers 3.84 .84 

Use assessment results to monitor student learning 3.83 .90 

Recognize unethical methods of using assessment 3.80 1 .04 

Determine proper difficulty of items 3.78 .89 

Develop a performance assessment rating scale 3.73 .90 

Identify the weakness of assessment methods 3.73 .74 

Use assessment results to organize a sound instructional plan 3.73 .93 

Write directions for assessment methods 3.72 .94 

Analyze the validity of test items 3.70 .90 

Select a representative sample of items to use for assessment purposes 3 .70 .90 

Communicate assessment results to parents 3 .67 1 .03 

Determine appropriate number of items of assessment method 3 .67 .86 

Identify structural problems in objective questions 3 .62 .84 

Examine items for gender bias 3 .52 1.00 

Compile a student portfolio 3 .38 1.15 

Score an essay 3 .36 1.05 

Interpret the scores of standardized achievement tests 3 .24 1.16 



Note. “ Based on scale . 1= not competent, 2 = slightly competent, 3 = moderately 
competent; 4 = very competent; 5 = extremely competent. 

** Three missing cases. 
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Table 4. 



Means° and Standard Deviations for Teacher Perceptions of Constraints to the 
Assessment Process ( n = 144) 



Item 


M 


SD 


1 decide what assessment methods 
to use in courses I teach 


4 .28 


.97 


Additional planning time would allow me to use 
assessment methods more effectively 


3 .94 


1 .10 


Equipment is available in my school for use 
in scoring tests 


3 .06 


1 .41 


Quality published assessment materials are hard 
to find 


3 .00 


1 .18 


Funds are available for buying published assessment 
materials 


2 .91 


1 .32 


College courses were of little help in preparing me 
to assess student learning 


2 .86 


1 .25 


In-service activities have helped develop 
my assessment skills 


2 .81 


1 .15 


I have assistance in preparing student assessment 
activities 


2 .75 


1 .37 


I do not have information on published 
assessment materials 


2 .69 


1 .20 



Note “ Based on scale: 1 = strongly disagree; 2 = disagree; 3 = neither agree nor 
disagree; 4 = agree; 5 = strongly agree. 
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Table 5. 



Multiole Regression Analysis 


for Use of Objective Paoer 


and Pencil 


Assessment 


Methods on Selected Variables fn = 144f“ 


Variable Standardized Beta Coefficients 


t 


Sig. 


Attitude 


.235 


2.698 


.007* * 


Constraints 


.101 


1.233 


.219 


Competence 


.248 


2.847 


.005* 


Gender 


.045 


.514 


.607 


Age 


-.023 


-.216 


.829 


Years of Teaching Exp. 


- .179 


-1.639 


.103 


Years of Related Work Exp. 


- .185 


-1.849 


.066 


Type of Program 


- .063 


-.771 


.442 


Educational Level 


.022 


.232 


.816 


Certification Route 


.056 


.612 


.541 



Note. Multiple correlation coefficient (R) = .463. 

Adjusted (“epsilon-squared”) =154. 

For Model: F = 3.558. 

*_g, < .05. 

Dependent Variable: objective paper and pencil assessment methods. 

* Three missing cases. 



Selected Variables 28 



Table 6. 

Multiple Regression Analysis for Use of Performance Assessment on Selected variables 
(n = 144)“ 



Variable Standardized Beta Coefficients t Sig. 



Attitude 


.370 


4.394 


.000* * 


Constraints 


.169 


2.143 


.03.4* 


Competence 


.134 


1.598 


.112 


Gender 


.172 


2.024 


.045* 


Age 


.005 


.056 


.955 


Years of Teaching Exp. 


-.133 


- 1.266 


.207 


Years of Related Work Exp. 


-.073 


- .761 


.447 


Type of Program 


-.075 


- .959 


.339 


Educational Level 


.050 


.544 


.587 


Certification Route 


.181 


2.053 


.042* 



Note. Multiple correlation coefficient (R) = .518. 
Adjusted (“epsilon-squared”) = .213. 

For Model: F = 4.790. 

* P< .05. 

Dependent Variable, performance assessment. 

“ Three missing cases. 
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Table 7, 



Multiple Regression Analysis for Use of Informal Observations on Selected Variables 
(n=144)“ 



Variable Standardized Beta Coefficients 


t 


Sig. 


Attitude 


.257 


2.966 


o 

o 


Constraints 


.067 


.835 


.405 


Competence 


.213 


2.467 


.014* 


Gender 


.166 


1.896 


.060 


Age 


-.101 


-.949 


.344 


Years of Teaching Exp. 


.028 


.263 


.793 


Years of Related Work Exp. 


.001 


.019 


.985 


Type of Program 


-.102 


-1.255 


.211 


Educational Level 


.137 


1.431 


.155 


Certification Route 


.184 


2.029 


.044* 



Note. Multiple correlation coefficient (R) = .474. 
Adjusted (“epsilon-squared”) = .165. 

For Model: F = 3.768. 

*^<.05. 

Dependent Variable, informal observations. 

“ Three missing cases. 
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Table 8. 

Multiple R egression Analysis for Use of Essay Items on Selected Variahips 
(n = 144)* 

Variable Standardized Beta Coefficients t 



Attitude 


-.025 


-.282 


.778 


Constraints 


.083 


.981 


.328 


Competence 


.267 


2.951 


.003* 


Gender 


-.098 


- 1 .076 


.283 


Age 


.027 


.247 


.805 


Years of Teaching Exp. 


-.165 


- 1.452 


.148 


Years of Related Work Exp. 


-.042 


- .403 


.687 


Type of Program 


-.110 


- 1.301 


.195 


Educational Level 


.081 


.809 


.420 


Certification Route 


.196 


2.064 


.041* 



Note. Multiple correlation coefficient (R) = .390. 
Adjusted (“epsilon-squared”) = .086. 

For Model; F = 2.33. 

*_P < .05. 

Dependent Variable: essay items. 

■* Three missing cases. 
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